SPECIFICATION 



TO ALL WHOM IT MAY CONCERN: 

BE IT KNOWN THAT I, TAMOTSU KAJIHARA, a 
citizen of Japan residing at Kanagawa , Japan have 
invented certain new and useful improvements in 

IMAGE PROCESSING APPARATUS AND METHOD 

of which the following is a specification:- 



BACKGROUND OF THE INVENTION 

1 . Field of the Invention 

The present invention generally relates to 
image processing apparatuses and image processing 
5 methods, and more particularly to an image processing 
apparatus and an image processing method for 
processing image data including a memory color, a 
program causing a computer to execute such a method, 
and a computer-readable recording medium on which 
10 such a program is recorded. 



2 . Description of the Related Art 
Progress in digital technology has made it 
common to output image data input through input 
15 apparatuses such as a digital camera and a network 
scanner from output apparatuses such as an ink- j et 
printer, a laser printer, and a digital proofer. 
Generally, image data input from an image data input 
apparatus is subjected to a variety of processing by 
20 an image processing apparatus or an image processing 
method causing the image processing apparatus to 
operate, and is output. 

Under such circumstances, a variety of 
images are processed so that a wide variety of 
25 requests as to image quality have been made to image 
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data . Particularly, the reproduction of colors 
referred to as memory colors such as skin color (skin 
tones), grass green, and sky blue, which are 
frequently seen in our daily life, is considered to 
5 be important. 

Particularly, the skin color of a human 
image is an image quality factor that attracts the 
most attention. Accordingly, techniques for 
correcting the skin color of a human image to an 

10 appropriate color have been disclosed. 

For instance, according to Japanese Laid- 
Open Patent Application No. 2000-242775 (Prior Art 1), 
a facial area is extracted from an image including a 
man, and color density information representing color 

15 and/or density (color density) around the facial area 
is detected. The color density of the facial area is 
adjusted based on the color density information. The 
adjusted image is reproduced so that the facial area 
of an appropriate color density unaffected by the 

20 color density around the facial area is perceived. 

Meanwhile, according to Japanese Laid-Open 
Patent Application No. 2001-186323 (Prior Art 2), a 
skin-colored area is extracted from a human image 
included in image data, and the skin color of the 

25 extracted skin-colored area is compared with a 
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prepared skin color correction target value to 
calculate a skin color correction. The color of the 
skin-colored area of the human image is corrected 
based on the calculated skin color correction. 
5 Thereby, the skin color of a man in an identification 
photograph can be made closer to an actual skin color. 

Further, it is well know that in recognizing 
the color and the density of a certain area, human 
vision is affected by color and density around the 

10 area, as described, for instance, in "Color 

Appearance Model," M. D. Fairchild, ADDISON— WESLEY , 
1998. That is, even if the density of an area 
remains the same, the area seems bright if it is dark 
around the area, and the area seems dark if it is 

15 bright around the area. 

For instance, when areas of the uniform gray 
density are observed against a background having the 
same gray density as those areas, those areas are 
perceived as having the same density. On the other 

20 hand, when the areas are observed against black and 
white, respectively, the areas are perceived as 
having different densities depending on their 
respective background colors. 

Accordingly, even when only a skin-colored 

25 area is processed to have an appropriate color and 
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density as described in Prior Art 2, a perceived 
facial area may be affected by the color and the 
density of the peripheral area to have an 
inappropriate color and density. 
5 Further, according to Prior Art 1, the color 

and the density of a skin-colored area is 
appropriately corrected considering the color and the 
density of its peripheral area. However, a 
preferable color is prevented from being reproduced 

10 in some cases. 

For instance, in the case of an image in 
which a skin-colored area exists in a preferable 
range as a whole and its peripheral area is tinged 
with blue as a whole, color correction to reduce 

15 blueness, that is, add yellowness, is performed on 
the entire image so as to perform correction to 
balance chromaticness over the entire image (so- 
called white balance correction) . 

The same color correction is performed on 

20 the skin-colored area so that the color of the skin- 
colored area may be offset from the center of a 
preferable skin color. A description is given, with 
reference to FIG. 1, of this problem. 

FIG. 1 is a diagram for illustrating color 

25 correction according to Prior Art 1. In FIG. 1, a 



skin-colored area and an area of a color other than a 
skin color (hereinafter referred to as a background 
area) that are extracted from image data are plotted 
in the CIELAB uniform color space. 

Referring to FIG. 1, the psychometric chroma 
coordinates of chromaticness are shown two- 
dimensionally . The horizontal axis represents 
chromaticness in the red-green direction (a*) , and 
the vertical axis represents chromaticness in the 
yellow-blue direction (b*). Point 5 represents the 
skin-colored area of the image data before correction, 
and Point Ri represents the chromaticness of the 
background area before correction. It is assumed 
that the chromaticness of the skin-colored area of 
the image data exists in a preferable skin-color 
range (indicated by hatching) . Naturally, in this 
case, the skin-colored area (point S) is perceived as 
a preferable skin color. 

However, in the case of image data tinged 
with blue over the entire image whose background area 
before -correction is indicated by Point R2 , correction 
to reduce blueness is performed on the entire image. 
Accordingly, correction to add yellowness to the 
entire image data is performed on the entire image. 
As a result, referring to FIG. 1, the background area 



is corrected from Point R2 to Point R2' . 

Meanwhile, the same color correction is 
performed on the image data of the skin-colored area 
so that the skin-colored area is corrected from Point 
S to Point S' . That is, although the colors are 
balanced over the entire image, the color of the 
skin-colored area is slightly more yellowish than a 
preferable skin color in the obtained image. That i 
the skin-colored area of this image data is not 
perceived as a preferable skin color. 

SUMMARY OF THE INVENTION 

Accordingly, it is a general object of the 
present invention to provide an image processing 
apparatus and method in which the above-described 
disadvantage is eliminated. 

A more specific object of the present 
invention is to provide an image processing apparatus 
and method that can correct the color of a memory 
color in image data in accordance with the peripheral 
area of the memory color and the characteristics of 
the image data so that a color perceived as 
preferable is always obtained. 

Another more specific object of the present 
invention is to provide a program for causing a 
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computer to execute such a method, and a computer- 
readable recording medium on which such a program is 
recorded . 

One or more of the above objects of the 
5 present invention are achieved by an apparatus for 

performing image processing on image data including a 
memory color, including: a characteristic value 
calculation part calculating a characteristic value 
from the image data ; an area extraction part 

10 extracting a specific color area of a specific color 
and a background area thereof from the image data , 
using the characteristic value and color information 
of the memory color stored in a storage part; a 
target value calculation part calculating a target 

15 value for correcting a preferable color of the 

specific color from the characteristic value, the 
extracted specific color and background areas, and 
color information of the preferable color stored in 
the storage part; and an image data color correction 

20 part calculating a correction for correcting the 

preferable color based on the calculated target value 
and the characteristic value, and correcting a color 
of the specific color area based on the calculated 
correction and the characteristic value. 

25 One or more of the above objects of the 
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present invention are also achieved by an apparatus 
for performing image processing on image data 
including a memory color, including: an area 
extraction part extracting a specific color area of a 
5 specific color and a background area thereof from the 
image data, using color information of the memory 
color stored in a storage part; a characteristic 
value calculation part calculating a characteristic 
value of at least one of the specific color area and 

10 the background area; a target value calculation part 
calculating a target value and a correction for 
correcting a preferable color of the specific color 
from a result of the calculation by the 
characteristic value calculation part and color 

15 information of the preferable color stored in the 
storage part; and an image data correction part 
correcting a color of the specific color area based 
on the correction calculated by the target value 
calculation part. 

20 One or more of the above objects of the 

present invention are also achieved by an apparatus 
for performing image processing on image data 
including a human image, including: an area 
extraction part extracting an area of a skin color 

25 and a background area thereof from the image data, 
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using color information of a memory color stored in a 
storage part; a characteristic value calculation part 
calculating a characteristic value of at least one of 
the area of the skin color and the background area; a 
5 target value calculation part calculating a target 
value and a correction for correcting a preferable 
color of the skin color from a result of the 
calculation by the characteristic value calculation 
part and color information of the preferable color 

10 stored in the storage part; and an image data 

correction part correcting a color of the area of the 
skin color based on the correction calculated by the 
target value calculation part. 

One or more of the above objects of the 

15 present invention are achieved by a method of 

performing image processing on image data including a 
memory color, the method including the steps of: (a) 
calculating a characteristic value from the image 
data; (b) extracting a specific color area of a 

20 specific color and a background area thereof from the 
image data, using the characteristic value and color 
information of the memory color stored in a storage 
part; (c) calculating a target value for correcting a 
preferable color of the specific color from the 

25 characteristic value, the extracted specific color 



and background areas, and color information of the 
preferable color stored in the storage part; and (d) 
calculating a correction for correcting the 
preferable color based on the calculated target value 
and the characteristic value, and correcting a color 
of the specific color area based on the calculated 
correction and the characteristic value. 

One or more of the above objects of the 
present invention are also achieved by a method of 
performing image processing on image data including a 
memory color, including the steps of: (a) extracting 
a specific color area of a specific color and a 
background area thereof from the image data, using 
color information of the memory color stored in a 
storage part; (b) calculating a characteristic value 
of at least one of the specific color 5 area and the 
background area; (c) calculating a target value and a 
correction for correcting a preferable color of the 
specific color from a result of the calculation by 
the step (b) and color information of the preferable 
color stored in the storage part; and (d) correcting 
a color of the specific color area based on the 
correction calculated by the step (c) . 

One or more of the above objects of the 
present invention are also achieved by a method of 



performing image processing on image data including a 
human image, including the steps of: (a) extracting 
an area of a skin color and a background area thereof 
from the image data, using color information of a 
memory color stored in a storage part; (b) 
calculating a characteristic value of at least one of 
the area of the skin color and the background area; 
(c) calculating a target value and a correction for 
correcting a preferable color of the skin color from 
a result of the calculation by the step (b) and color 
information of the preferable color stored in the 
storage part; and (d) correcting a color of the area 
of the skin color based on the correction calculated 
by the step (c) . 

One or more of the above objects of the 
present invention are also achieved by a program for 
causing a computer to execute a method according to 
the present invention. 

One or more of the above obj ects of the 
pr.esent invention are further achieved by a computer- 
readable recording medium on which a program for 
causing a computer to execute a method according to 
the present invention is recorded. 
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BRIEF DESCRIPTION OF THE DRAWINGS 

Other objects, features and advantages of 
the present invention will become more apparent from 
the following detailed description when read in 
5 conjunction with the accompanying drawings, in which: 

FIG. 1 is a diagram for illustrating a 
problem in the correction of a preferable skin color 
according to prior art; 

FIG. 2 is a block diagram showing a 
10 functional configuration of an image processing 
apparatus according to a first embodiment of the 
present invention; 

FIG. 3 is a diagram for illustrating the 
color information of a memory (skin) color and the 
15 setting of a preferable skin color according to the 
first embodiment of the present invention; 

FIGS. 4A and 4B are diagrams for 
illustrating the splitting of the original image data 
into areas according to the first embodiment of the 
20 present invention; 

FIG. 5 is a diagram showing a characteristic 
data table according to the first embodiment of the 
present invention ; 

FIG. 6 is a diagram showing the results of 
25 the mapping of chromaticity coordinates in each area 



of FIG. 4B into the CIELAB uniform color space 
according to the first embodiment of the present 
invention ; 

FIG. 7 is a diagram for illustrating a 
method of calculating a correction according to the 
first embodiment of the present invention; 

FIG. 8 is a diagram for illustrating the 
relationship between the correction and the distance 
between a specific color area to be subjected to 
correction and its background area according to the 
first embodiment of the present invention; 

FIG. 9 is a block diagram showing a 
functional configuration of the image processing 
apparatus according to a second embodiment of the 
present invention ; 

FIG. 10 is a diagram showing an Exif-based 
file structure storing the attribute information of 
the original image data according to the second 
embodiment of the present invention; 

FIG. 11 is a diagram showing a correction 
factor table corresponding to a scene capture type 
according to the second embodiment of the present 
invention ; 

FIG. 12 is a flowchart showing a procedure 
for correction in the image processing apparatus 



according to the first and second embodiments of the 
present invention ; 

FIG. 13 is a block diagram showing a 
configuration of the image processing apparatus 
according to a third embodiment of the present 
invention ; 

FIG. 14 is a diagram for illustrating the 
calculation of a characteristic value, a target value, 
and a correction according to the third embodiment of 
the present invention; 

FIGS. 15A and 15B are diagrams for 
illustrating an extracted skin-colored area and 
background area according to the third embodiment of 
the present invention; 

FIG. 16 is a schematic graph of a frequency 
distribution of luminance in a luminance plane 
according to the third embodiment of the present 
invention ; 

FIG. 17 is a schematic graph of a frequency 
distribution of chroma in a chroma plane according to 
the third embodiment of the present invention; 

FIG. 18 is a graph for illustrating the 
calculation of the correction from the characteristic 
value relating to the luminance of the background 
area according to the third embodiment of the present 



invention ; 

FIG. 19 is a graph for illustrating the 
calculation of the correction from the characteristic 
value relating to the chroma of the skin-colored area 
according to the third embodiment of the present 
invention ; 

FIG. 20 is a block diagram showing a 
configuration of the image processing apparatus 
according to a fourth embodiment of the present 
invention ; 

FIG. 21 is a diagram showing a correction 
factor table corresponding to a scene capture type 
according to the fourth embodiment of the present 
invention ; and 

FIG. 22 is a flowchart showing a procedure 
for correction in the image processing apparatus 
according to the third and fourth embodiments of the 
present invention. 

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS 

A description is given, with reference to 
the accompanying drawings, of embodiments of the 
present invention . 

[First Embodiment] 

FIG. 2 is a block diagram showing a 



functional configuration of an image processing 
apparatus according to a first embodiment of the 
present invention. Referring to FIG. 2, image data 
is input from an image data input part 10 , subjected 
to color correction in a color correction unit 100, 
and output from an image data output part 30. 

The color correction unit 100 includes a 
unit controller 101, a characteristic value 
calculation part 103, an area extraction part 105, a 
target value calculation part 107, a storage part 109 
and an image data correction part 111. 

The unit controller 101 controls the entire 
color correction unit 100. The characteristic value 
calculation part 103 calculates the characteristic 
values of the image data input from the image input 
part 10, which is hereinafter also referred to as the 
original image data, and outputs the characteristic 
value table of the original image data. The area 
extraction part 105 extracts a specific color area to 
be subjected to color correction and its background 
area included in the original image data, referring 
to the color information of a memory color stored in 
the storage part 109. The target value calculation 
part 107 calculates the target value of a preferable 
color after color correction for the specific color 



from the color information of the extracted specific 
color area and background area and preferable color 
information for the specific color obtained from the 
storage part 109. The image data correction part 111 
calculates a correction (amount) in accordance with 
the calculated target value of the preferable memory 
color and characteristic values obtained from the 
characteristic value table of the original image data. 
Then, the image data correction part 111 obtains 
image data by performing color correction on the 
original image data by the calculated amount. The 
obtained image data is hereinafter also referred to 
as corrected image data. 

The image data input part 10 may be 
connected to a network such as the Internet through a 
network interface provided to the image processing 
apparatus so that image data can be input from 
outside via the network.. Alternatively, image data 
may be input from apparatuses connected to the image 
processing apparatus, such as a hardware disk drive, 
a flexible disk drive, an optical recording medium 
reader, a scanner, and a digital camera. 

When the input image data is an analog 
signal, the image data input part 10 converts the 
input image data into a digital signal. Then, the 



image data input 10 performs predetermined digital 
processing on the input image data, and outputs the 
image data to the color correction unit 100 as 
digital signals of R (red) , G (green) , and B (blue) . 

In this embodiment, the RGB color space is 
employed for the input signal, and the Lab color 
space is used for characteristic value calculation 
and area extraction. However, the present invention 
is not limited to these color spaces, and may also 
employ the YCC color space or the Luv color space. 

The storage part 109 stores the chromaticity 
information of memory colors and the color 
information of a preferable color (preferable color 
information) for each memory color. The chromaticity 
information is stored as extraction conditions for 
extracting a specified memory color from the original 
image data. The memory colors are colors memorized 
by a man, such as the colors of skin, sky, ocean, 
river, grass and trees, and snow. 

Next, a description is given of a method of 
setting the color of skin (skin color) of those 
memory colors, and a method of setting the color 
information of a preferable skin color. 

Method of setting a skin color : 

As a method of setting a skin color, for 
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instance, such ranges as defined in "Skin Tone Color" 
by Japan Color Research Institute may be prestored as 
follows : 



Hue (H) : 1YR-9YR 
Chroma (C) : 3, 4, and 5 
Value (V) : 4 . 0-8 . 5 



The value of HV/C employed herein is 
converted into HV/C (HVCjis) and the corresponding xy 
chromaticity coordinates by: 

x jis —f fajis • Vjis # Cjjs ) 

yjis =/ (h j1s • Vjis •Cjjs) - ( i > 



as illustrated in "Colour specification-Specification 
according to their three attributes" in JIS (Japanese 
Industrial Standards Z 8721. 

Next, values in an appropriate uniform color 
space are obtained from the obtained x n y, and V 
(*Yjis) coordinates by: 



yjis 
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Y = V 



MS 



(2) 



Z = 



\ x jjs ym 
yjis 



• v, 



MS 



I* = 116«(— ) 3 -16 = 116. 7.787 •(— + —) -16 



^ = 200.J/(^)-/(-|-)i ... (3) 



/ & = i0 = 7-787 . (I- + 
T T T. 116 



In this embodiment, the range of a skin- 
10 colored area is calculated by calculating the 

tristimulus values (X, Y, Z) of the CIE 1931 standard 
colorimetric system (CIEXYZ 1931) and the CIELAB 
values. FIG. 3 is a diagram showing a skin-color 
range. An ellipsoid including 90 % of the Lab values 
15 of the obtained points may be defined, and the range 

of the ellipsoid may be defined as a skin-color range. 

Letting the coordinates of the intersections 
Ui and U 2 of the periphery of the above-described 
skin-color range and the line segments connecting the 
20 periphery and the origin be (L* (Ui) , a* (Ui) , b* (Ui) ) 
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and (L*(U 2 ), a* (U 2 ) , b* (U 2 ) ) , respectively , and 
letting the coordinates of the intersections U 3 and U 4 
of the periphery of the above-described skin-color 
range and the line segments connecting the center of 
the ellipsoid and the origin be (L* (U 3 ) , a* (U 3 ) , 
b*(U 3 )) and (L*(U 4 ), a* (U 4 ) , b* (U 4 ) ) , respectively, it 
is possible to set the maximum and minimum hue and 
chroma values of the four points as the skin-color 
range . 

Further, at the time of the conversion into 
values in an appropriate equivalent color space, it 
is required to set an environment for color 
observation. For instance, in Equation (1), the 
color observation environment is limited to an 
observation environment at a light source. Therefore 
a skin-colored area in another observation 
environment cannot be defined. 

However, a user observes image data in a 
variety of environments. Therefore, it is desirable 
to convert the CIEXYZ 1931 tristimulus values 
calculated from Equation (1) by applying, for 
instance, a color appearance model such as the von 
Kries model or CIECAM in accordance with color 
appearances by various light sources (under various 
lighting conditions) . It is necessary to specify 



observation environment parameters to perform the 
conversion. It is suitable that characteristic 
values are preset for the parameters, and are changed 
when the user specifies the values of the parameters. 

Method of setting a preferable skin color : 
Next, a description is given of a method of 
setting the color information of a preferable skin 
color. 

First, a plurality of image data items each 
containing a skin color are collected, and one or 
more of the image data items are subjectively- 
selected through subjective close examination. Then, 
subjective evaluation as to whether the selected 
image data is "preferable" or "not preferable" is 
performed . 

The chromaticity coordinates of the skin 
color of image data determined as having a high 
subjective evaluation point are employed. The 
principal components of the employed chromaticity 
coordinates are analyzed using Equation (4) so as to 
calculate the first, second, and third principal 
component vectors and their respective eigen values . 
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where Pi , P2 , and P3 are the first, second, and third 
components, respectively, Aj , A2, and A3 are the eigen 
values of the first, second, and third components, 
respectively, and pn through P33 are constants. 

Letting the center of the calculated 
ellipsoid be the center coordinates R of a preferable 
skin color, and setting the square roots of the eigen 
values of the respective principal components as 
tolerance, a preferable skin-color range as shown in 
FIG. 3 is set. 

A memory color and preferable color 
information therefor that are set as described above 
are prestored in the storage part 109. When the 
listed objects of memory colors in pages 7-18 of 
"Study of Phenomenal Characteristics of Memory Colors 
(First Report)" by Kazuyuki Natori and Gentaro Omi , 
Color Study, the Color Science Association of Japan, 
2000, are employed as memory colors, for instance, 
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the same processing is applicable. 

Further, chromaticity is not limited to that 
of the Lab color space. The chromaticity of the YCC 
color space, xy chromaticity, and chromaticity using 
5 an RGB ratio are also employable. 

The specific color subjected to correction 
in the color correction unit 100 may be specified by 
a user as follows . 

First, a selection button for color 
10 specification is selected from a menu displayed on 

the screen of an image display unit (not graphically 
represented) . Next, a message asking which memory 
color to employ as a specific color is successively 
displayed on the screen with four selection buttons 
15 of Memory Color 1, Memory Color 2, User Specification 
1, and User Specification 2 being displayed below the 
message. The user can select one of the four 
selection buttons . 

By selecting a desired memory color from the 
20 four selection buttons, the memory color 

corresponding to the selected button is read out from 
the storage part 109 as the specific color. 

Predetermined memory colors such as a human 
skin color and the color of sky may be correlated 
25 with Memory Color 1 and Memory Color 2, respectively, 
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and memory colors previously specified by the user 
may be correlated with User Specification 1 and User . 
Specification 2. For instance, the color of water 
and the color of grass and trees may be correlated 
5 with User Specification 1 and User Specification 2, 
respectively. Thus, the user can set specific colors 
in memory colors based on the original image data. 

In the following description, the specific 
color selected by the user is the memory color of a 

10 human skin color. However, the present invention is 
also applicable in the case of other memory colors. 
Further, the area formed of the specific color in the 
original image data is referred to as a specific 
color area. When the memory color of a human skin 

15 color is selected as the specific color, the specific 

color area is also referred to as a skin-colored area. 

The image data output part 30 outputs the 
image data subjected to color correction in the color 
correction unit 100 to a printer, a multi-function 

20 apparatus, a facsimile machine, or a storage device 
such as a hard disk unit or an optical recording 
medium recorder. The image data output part 30 may 
output the corrected image data to outside via a 
network . 

25 Next, an expatiation is given of each of the 
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components of the color correction unit 100. 

Characteristic value calculation part 103 : 
The characteristic value calculation part 

103 calculates the characteristic values of the image 
5 data supplied from the image data input part 10 , and 

outputs a characteristic value table. 

First, the R, G, and B data (R, G, B) of 

each pixel of the original image data input from the 

image data input part 10 is converted into 
10 chromaticity coordinates (L* , a*, b*) in the CIELAB 

uniform color space by: 



L* 




«n 


a l2 


«13 
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a 22 
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fl 33. 


B 



15 where an through a 3 3 are conversion factors formed of 
constants . 

Next, a description is given of a method of 
calculating the characteristic values of the original 
image data from the obtained chromaticity coordinates . 
20 The original image data is split into areas 

(regions) in accordance with its image 
characteristics , and the characteristic values are 
calculated area by area. As the method of splitting 
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the original image data into areas, a variety of 
methods is applicable, such as a method of splitting 
image data into areas recursively from the 
statistical information of the image data 
5 ( " Structured Description of Color-Image Data Region 
Splitting" by Yu-ichi Ota, Takeo Kanade, and 
Toshiyuki Sakai, Information Processing, Information 
Processing Society of Japan, December 1978, Vol. 19, 
No. 12, pp. 1130-1136) or a splitting method using 

10 the statistical information of an image ("Picture 

Segmentation Using A Recursive Segmentation Splitting 
Method" by Ron Ohlander et al . , Comp . Graph, and 
Image Proc., pp. 313-333). 

In this embodiment, a splitting method to 

15 which the K-means algorithm is applied is employed. 

For instance, image data as shown in FIG. 4A is split 
into areas so that an image as shown in FIG. 4B is 
obtained. The image of FIG. 4B is subjected to 
contrast correction to be represented by shading- 

20 information so as to make it easy to understand the 
results of the area splitting. 

The details of this splitting method are 
described in the above-mentioned documents. The 
outline is as follows. 

25 First, the entire screen of the original 



image data is split into preset sub-areas (regions) 
so that initial areas are generated. Then, a 
statistical information table is calculated (created) 
for each initial area. In this embodiment, the 
original image data is split into 25 (vertically five 
times horizontally five) areas (5x5= 25) . However 
only five of the 25 sub-areas are shown in FIG. 4B 
for convenience of description. 

At this point, each statistical information 
table is composed of the arithmetic average (L* ave , 
<3*ave, b*ave) of the color information Lab values (!>*, 
a*, b*) of the corresponding initial area and the 
arithmetic average (lave, Jave) of the position 
information (i, J) of the corresponding initial area. 
The arithmetic average (L* ave , a* ave , b*ave) and the 
arithmetic average (lave, Jave) are calculated from: 



V W V W V w 

ZZ^*~ E2>*. EE*** 

T * _ t=\ u=\ * _ f=l u=l L * _ /=! «=1 

ave r ave r ° ave 

VX W VXW VX W 

V W V w 

EE'~ EE^ 

I m = - " =1 , J m = J ^ -(6) 

vxw vxw 



where v and w are the number of vertical and 
horizontal pixels [pix] of the initial area, 
respectively, L* t u, &*tu, and b*tu are Lab values at a 
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pixel position [t, u] , and Itu, and J tu are position 
information at a pixel position [t, u] . 

Next, the relative distance D(h) between the 
chromaticity coordinate values (L*, a*, b*) and the 
position information (i, j) of the image data and 
each statistical information table is calculated by a 
sub-area search using Equation (7) so as to determine 
in which of the 25 sub-areas the relative distance 
D(h) is minimized. 



- (7) 

where Qi and Q2 are constants, which are 1 ( Qi = Q2 = 
1) in this embodiment. 

This sub-area search is repeated as many 
times as the number of sub-areas (25 times in this 
embodiment) to search for the number h of the sub- 
area that minimizes D(h), This h represents 
sequential serial numbers for the sub-areas, and 
corresponds to the subscript h of Hh in the table of 
FIG. 5. 

The contents of the statistical information 



table of the searched-out sub-area and the number of 
the sub-area to which the pixel (i, j) originally 
belongs are rewritten, thereby rewriting the 
statistical information tables of the sub-area to 
which the pixel (i, J) originally belongs and the 
searched-out sub-area. This operation is repeated as 
many times as the number of pixels of the image data. 
In this embodiment, this operation is repeated 
(i+1) x ( times . 

After repeating the above-described 
processing a predetermined number of times, the image 
data split into areas as shown in FIG. 4B is obtained. 

Finally, the characteristic value table of 
the original image data is created based on the 
information on the sub-areas obtained in their 
respective statistic information tables, employing 
the central value (L *ave , &*ave, &*ave ) of the 
chromaticity coordinates of the original image data 
and the percentage As [%] of an area occupied by a 
sub-area in the area of the original image data of 
each sub-area as the characteristic values of the 
original image data. 

In this embodiment, the average {L* ave , a* a ve, 
b*ave) is employed as characteristic values of the 
original image data (image data before correction) . 
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However, a variety of statistics similar to the 
average, such as a standard deviation, a median, and 
a mode, are also employable. 

As a result of the above-described 
5 processing, the characteristic value table is 

obtained from the original image data as shown in FIG. 
5 . 

Area Extraction Part 105 : 

The area extraction part 105 extracts the 
10 specific color area (skin-colored area) and its 

peripheral background area of the original image data 
supplied from the image data input part 10 from the 
calculated characteristic value table and the color 
information of a memory color prestored in the 
15 storage part 109. 

In order to perform the extraction, the R, G, 
and B original image data (R, G, B) is subjected to 
the conversion of Equation (5) sub-area by sub-area 
as in the characteristic value calculation part 103 
20 so as to be mapped into the CIELAB uniform color 
space . 

A plurality of points onto which the 
original image data is mapped are defined as Hh 
(L* (Hh) , a*(H h ), b*{H h )), where h is the number of 
25 sub-areas (h = 1, ...) . Of the points, those belonging 
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to a range in the chromaticity coordinates of the 
specific color (skin color) which range is preset in 
the storage part 109 are determined to be the 
specific color area (skin-colored area) . 

Next, a description is given, with reference 
to FIG. 6, of a method of extracting the specific 
color area (skin-colored area) . 

FIG. 6 is a diagram showing the results of 
the mapping of chromaticity coordinates in each split 
area (sub-area) of the original image data of FIGS. 
4A and 4B into the CIELAB uniform color space. A 
range denoted as a "specific color range" in FIG. 6 
indicates the range of the chromaticity coordinates 
of the specific color (skin color) stored in the 
storage part 109. The points of the original image 
data mapped within this range correspond to the 
specific color. 

Points H h in FIG. 6 correspond to the 
numbers h of the sub-areas of the original image data 
Points H4 and H5 are the chromaticity coordinates of 
the image data to be extracted as the specific color 
area (skin-colored area) . 

The specific color area (skin-colored area) 
extracted at this point includes the two points H4 and 
H5 . In this case, a point (or an area) having the 



higher (highest) percentage As [%] is determined to 
be the specific • color area (skin-colored area) . 
Accordingly, Point H 4 is determined as the specific 
color area (skin-colored area) . 

The area having the highest percentage As 
[%] is determined based on the percentages As [%] of 
the split areas (sub-areas) entered in the previously 
created characteristic value table. 

Likewise, of the three areas Hi, H2 , and H3 
other than the specific color area (skin-colored 
area) , the area Hi having the highest percentage As 
[%] is selected as the background area of the 
specific color area. 

Further, in the case of extracting the 
specific color area and its background area, if the 
specific color area or the background area is smaller 
than a predetermined area size, it may be determined 
that the specific color area or the background area 
is not to be employed for correction. As a result, 
an appropriate area may be extracted. A percentage 
As for the predetermined area size is suitably less 
than or equal to 5 to 10 %. 

Target value calculation part 107 : 

The target value calculation part 107 
calculates the target value of a preferable color 



which serves as a target of correction for the 
specific color area (skin-colored area) of the 
original image data. A description is given, with 
reference to FIG. 7, of a method of calculating the 
target value. Referring to FIG. 7, R indicates the 
chromaticity coordinates of the center of a 
preferable specific color area (preferable skin- 
colored area) , and S indicates the characteristic 
values of the specific color area (skin-colored area) 
of the original image data. The above-described 
Point H 4 corresponds to Point S. The chromaticity 
coordinates R' of the specific color (skin color) 
that serve as the target value are calculated from 
Point Q of the background area (corresponding to the 
above-described Point Hi) and Point R. 

In the following, the target value is 
calculated by solving the equations of straight lines 
connecting the points on chromaticity coordinates 
using two-dimensional space coordinates. However, 
the method of calculating the target value is not 
limited to this method. For instance, three- 
dimensional principal components are applicable as 
below-described three-dimensional components . 
Further, it is possible to calculate the equation of 
a straight line using vector components. 
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First, letting the coordinates of Point O 
(origin) and Point Q (background) be 0(a*(o), b* (o) ) 
and Q(a*(q) f b*(q)) f respectively, a straight line 1 
that has the same slope as a segment passing through 
5 Points 0 and Q and passes through Point R 
(preferable) is expressed by: 



b*= b y- b yM a*-a*(r)} + b*(r) -(8) 
a * (g ) - a * (o) 

10 At this point, the coordinates of Point R 

are R(a*(r), b*(r)), and the intersection of the 
straight line 1 and an ellipsoid E formed of the 
first and second principal components of the 
preferable specific color area (preferable skin- 

15 colored area) is calculated as R' (target) . The 
obtained coordinates are defined as R'(a*(r'), 
b*(r')) . The ellipsoid E is defined by: 

... (9) 

■•■(10) 
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where Pi and P^.are the first and second principal 



components, respectively, and ^2 are the eigen 
values of the first and second principal components, 
respectively, and pn through P22 are constants. 

A straight line m connecting Point S(a*(s), 
b*(s)) of the characteristic values of the original 
image data and the target value R' is calculated 
from: 

a*(r )-a*U) 

and the control point S' (target) of the image data 
is set at a predetermined point on the straight line 
in . 

At this point, a correction Th from Point S 
to Point S' is calculated from the characteristic 
values of the image, which is described below with 
reference to the image data correction part 111. 
Image data correction part 111 : 
The image data correction part 111 
calculates the correction Th in accordance with the 
target value of the preferable color for the specific 
color calculated in the target value calculation part 
107 and the characteristic values obtained from the 
characteristic value table of the original image data. 
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Then, the image data correction part 111 obtains the 
corrected image data by performing color correction 
on the original image data by the correction Th . 

A correction according to the percentage As 
[%] of the specific color area (skin-colored area) in 
the original image data is stored in a memory such as 
an LUT (look-up table) as the correction Th . 

Alternatively, a function such as Equation 
(12) may be defined so as to suitably calculate the 
correction Th using the percentage As obtained from 
the characteristic value table. 



, a(l-r)*(ASY ...0<AS s <a 
" [l-Cl-ay-'.a-AS,)' ~.a<AS s <i "' { 



where AS S is the percentage As of the specific color 
(skin color), and a and r are constants of 0 to 1 . 

Next, a description is given, with reference 
to FIG. 7 , of a method of performing correction on 
the entire image. 

The original image data is converted from 
the Lab values to the values Pi and P2 by Equation 
(10) . 

Then, a point Si of the original image data 
is corrected to a point Si' in accordance with the 



distance from the background area. A correction STh 
for this correction is expressed by Equation (13), 
letting the distance between Point Q of the 
background area and Point S of the image be | QS | and 
the distance between Point S and Point S' be Th . 



Srh = j(P 1 STh) 2 +(P 2 STh) 2 
P.STh = Th* cos &i 



OS, 



QS 



P 2 $Th = Tff cos 0 2 • 



OS, 



\QS\ 



(13) 



where 0i and 0 2 are angles that the first and second 
principal components, respectively, form with respect 
to the straight line m, and is the distance 

between Points Q and S. 

A description is given, with reference to 
FIG. 8, of the detailed movement of this correction. 
FIG. 8 shows positions of color correction before and 
after the correction. As described above, Point S 

(image) close to the specific color area (skin- 
colored area) is corrected to Point S' (target) . In 
the drawing, an arrow indicates the magnitude 

(amount) of correction. Likewise, a point Si is moved 
by a correction Thi (composed of PiSThx and P2STI11) and 



corrected to a point Si'. Further, a point S2, which 
is close to the background area, is moved by a 
correction Th 2 (composed of PiSTh 2 and P2STI12) and 
corrected to a point S2' . 

The comparison of these corrections show 
that as a point moves away from the specific color 
area (skin-colored area) and comes closer to the 
background area as S— >Sj— >S 2f a correction for the 
point decreases as | Th \ > \Th 2 \ > \Th 2 \. 

By this correction, the specific color area 
(skin-colored area) of the image is corrected to a 
preferable color while the color of the background 
area is scarcely corrected. Accordingly, only the 
specific color (skin color) may be suitably corrected 
without losing the balance of the entire image. 

Finally, Pi and P2 are inversely converted 
into Lab values, and are further inversely converted 
into R, G, and B values by: 

- (14) 
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As a result, the corrected image data is obtained, 
and is supplied to the image data output part 30. 

According to this embodiment, a linear 
primary expression that inversely converts Lab values 
into R, G, and B values is employed. However, it is 
also possible to store this in an LUT and coverts Lab 
values into R, G, and B values by an interpolation 
operation. Alternatively, an LUT that converts Lab 
values into density is also employable. 

The same processing as described above is 
performable in the case of correcting another memory 
color. In this case, the a-b plane may be split into 
appropriate hues (areas) , and the correction of the 
memory color may be controlled in each area to which 
the memory color belongs. 

[Second Embodiment] 

According to a second embodiment, a 
correction obtained in the same way as in the first 
embodiment is subjected to image processing so that a 
preferable color is always obtained in accordance 
with the attribute information of the original image 
data obtained from outside. 

FIG. 9 is a block diagram showing a 



-42- 



functional configuration of the image processing 
apparatus according to the second embodiment of the 
present invention. Referring to FIG. 9, image data 
is input from the image data input part 10, subjected 
5 to color correction in the color correction unit 100, 
and output from the image data output part 30. 

The color correction unit 100 includes the 
unit controller 101, the characteristic value 
calculation part 103, the area extraction part 105, 

10 the target value calculation part 107, the storage 

part 109, an attribute information obtaining part 112, 
and the image data correction part 111. 

The unit controller 101 controls the entire 
color correction unit 100. The characteristic value 

15 calculation part 103 calculates the characteristic 
values of the image data input from the image input 
part 10, which is hereinafter also referred to as the 
original image data, and outputs the characteristic 
value table of the original image data. The area 

20 extraction part 105 extracts a specific color area to 
be subjected to color correction and its background 
area included in the original image data, referring 
to the color information of a memory color stored in 
the storage part 109. The target value calculation 

25 part 107 calculates the target value of a preferable 
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color after color correction for the specific color 
from the color information of the extracted specific 
color area and background area and preferable color 
information for the specific color obtained from the 
storage part 109. The attribute information 
obtaining part 112 obtains the attribute information 
of the original image data. The image data 
correction part 111 calculates a correction (amount) 
in accordance with the calculated target value of the 
preferable memory color and characteristic values 
obtained from the characteristic value table and the 
attribute information of the original image data. 
Then, the image data correction part 111 obtains 
image data by performing color correction on the 
original image data by the calculated amount. The 
obtained image data is hereinafter also referred to 
as corrected image data. 

In FIG. 9, the elements having the same 
functions as in the first embodiment (FIG. 2) are 
referred to by the same numerals, and with respect to 
those elements, only a difference from the first 
embodiment is described. 

The attribute information obtaining part 112 
reads out information added to the original image 
data supplied from the image data input part 10, and 
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supplies the read-out information to the image data 
correction part 111. Any file structure such as the 
Exif or DPOF format structure is employable as the 
data format of the read-out information. 
5 For instance, the attribute information 

obtaining part 112 obtains the attribute information 
of the original image data, which is formed of a file 
structure based on the Exif format as shown in FIG. 
10. Referring to FIG. 10, in the case of obtaining 

10 image attribute information on the shot scene type of 
the information on picture-taking conditions of the 
original image data, each integer value of the type 
(unsigned integer value) of the scene capture type in 
41990 of the tag number (HEX: hexadecimal) of the 

15 image is read. For instance, if the type of the 

scene capture type is 0 , it is shown that the taking 
of a picture has been performed under standard 
picture-taking conditions. If the type of the scene 
capture type is 2 , it is shown that a portrait (human 

20 image) has been captured. 

When the image attribute information 
attached to the original image data has been read , 
the image data correction part 111 obtains a 
correction factor a sce ne according to the image 

25 attribute information from a correction factor table 
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(LUT) . Then, the image data correction part 111 
multiplies a correction calculated in the same manner 
as in the first embodiment by the correction factor 
ctscene, and corrects the original image data using the 
5 multiplied correction as the final correction. 

When the image attribute information is the 
scene capture type, the correction factor table has a 
data structure such that the types of the scene 
capture type are correlated with their respective 
10 correction factors oc SC ene as shown in FIG. 11. 

For instance, when a portrait is captured 
(this case corresponds to Type 2 of FIG. 11) , the 
correction factor oc scene is 1.30. This value is higher 
than that for STANDARD (corresponding to Type 0 of 
15 FIG. 11) . This is because when a portrait is 

captured, a correction is increased since emphasis is 
put on the reproducibility of a skin-colored area in 
the image data. 

A method of calculating the correction is 
20 not limited to the above-described method that refers 
to the correction factor table. For instance, the 
correction factor cc S cene may be suitably calculated by: 



a \ {(1 -r).(A5 Jtt ,r} ...0<AS skin <a 
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where ' A5 s jcin is the percentage As of the specific color 
(skin color) , and a and r are constants of 0 to 1 . 

Further, the image attribute information is 
not limited to the scene capture type. Correction 
may be performed based on other image attribute 
information. For instance, in the above-described 
correction factor table or Equation (15), the 
correction factor oc scene may be set so that oc scene = 1 
when a flash is used and detected and obscene « 0 in the 
other cases. As a result, correction to set off a 
man in a night scene mode may be controlled. 

Further, the image data of the skin-colored 
area may be reproduced more preferably by increasing 

the correction factor cc SC ene if, of the image attribute 
information, the F number is great or the shutter 
speed is low. 

Further, it is also possible to perform 
correction in accordance with the magnitude of white 
balance correction. The R, G, and B values of the 
skin-colored area have their respective averages 
compared with those of the entire image. The 
differences are set in an LUT so that correction is 
performed based on the set differences. 

That is, even when the image of a man is 



captured with considerably poor white balance, the 
skin color of the man is set within a preferable 
range. Accordingly, the image data of the skin- 
colored area can be reproduced more preferably. 

According to the configuration of the second 
embodiment, a specific one of memory colors (the 
specific color or the human skin color) desired by a 
user may be corrected in accordance with its 
peripheral area so that a preferable color is always 
obtained . 

Next, a description is given, with reference 
to FIG. 12, of a procedure for correcting the 
original image data in the image processing apparatus 
according to the first and second embodiments. 

First, in step S10 of FIG. 12, image data is 
input from outside via a network or from an apparatus 
such as a hardware disk drive, a flexible disk drive, 
an optical recording medium reader, a scanner, or a 
digital camera. When the input image data is an 
analog signal, the image data is converted into a 
digital signal. Thereafter, the image data is 
subjected to predetermined digital processing, and is 
output as digital signals of R (red) , G (green) , and 
B (blue) . 

Next, in step Sll, the R, G, and B data (R, 
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G, B) of each input pixel is converted into 
chromaticity coordinates (L* , a*, b*) in the CIELAB 
uniform color space by Equation (5). Using. the 
obtained chromaticity coordinates, the original image 
data is split into sub-areas recursively from its 
statistical information. Then, the central value 
(L*ave, a*ave, b*ave) of the chromaticity coordinates of 
the original image data and the percentage As [%] of 
an area occupied by a sub-area in the area of the 
original image data are calculated as characteristic 
values for each sub-area, and are stored in the 
characteristic value table. 

In step S12, the original image data (R, G, 
B) is subjected to the conversion of Equation (5) 
sub-area by sub-area to be mapped into the CIELAB 
uniform color space. Of the mapped points Hh, those 
belonging to a range in the chromaticity coordinates 
of the specific color (skin color) which range is 
stored in the storage part 109 are determined to be 
the specific color area (skin-colored area) . The 
background area of the specific color area is 
selected from the rest of the mapped points. 

If the percentage As [%] of a sub-area 
entered in the characteristic value table is smaller 
than a predetermined area size (a percentage As [%] 
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less than or equal to 5 to 10 %) , the sub-area is not 
to be selected as the specific color area (skin- 
colored area) or the background area. 

Then, in step S13, the target value of a 
5 preferable color serving as a target of correction 
for the specific color area (skin-colored area) of 
the original image data is calculated. 

Referring to FIG. 7, first, the straight 
line 1 that has the same slope as the segment passing 

10 through Points O (origin) and Q (background) and 
passes through Point R (preferable) is obtained 
(Equation (8)), and the intersection of the straight 
line 1 and the ellipsoid E (Equations (9) and (10) ) 
formed of the first and second principal components 

15 of the preferable specific color area (preferable 
skin-colored area) is calculated as R' (target) . 

Next, the straight line m connecting Point S 
of the characteristic values of the original image 
data and the target value R' is calculated from 

20 Equation (11) , and the control point S' (target) of 

the image data is set at a predetermined point on the 
straight line in. 

The correction Th from Point S to Point S' 
is calculated in accordance with the percentage As of 

25 the specific color area (skin-colored area) using an 
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LUT or a function such as Equation (12) . 

In step S14, a correction by which the 
original image data is corrected is calculated based 
on the calculated target value of the preferable 
5 color and the characteristic values for the specific 
color area (skin-colored area) obtained from the 
characteristic value table. The original image data 
is subjected to color correction by the calculated 
correction. Thus, the corrected image data is 
10 obtained. 

Referring to FIG. 7, the original image data 
is converted from the Lab values to the values Pi and 
P2 by Equation (10) . Then, letting the distance 
between Point Q of the background area and Point S of 

15 the image be | QS | and a correction at Point S be Th , 
the correction STh for Point S± of the original image 
data is calculated from Equation (13) . Point S is 
corrected to Point S±' using the correction STh . 

If the image attribute information added to 

20 the original image data has been obtained at this 

point, the correction factor cc S cene according to the 
image attribute information is obtained from the 
correction factor table or calculated from Equation 
(15) . The correction Th and the correction STh are 

25 multiplied by the correction factor obscene to be 



determined as the final corrections. 

Finally, using Equation (14), the values Pi 
and P2 are inversely converted into Lab values, and 
are further inversely converted into R, G, and B 
values. Thus, the (inversely converted) corrected 
image data is obtained. 

In step S15, the corrected image data 
obtained by performing color correction on the 
original image data is output to a printer, a multi- 
function apparatus, a facsimile machine, or a storage 
device such as a hard disk unit or an optical 
recording medium recorder. Alternatively, the 
corrected image data may be output to outside via a 
network . 

[Third Embodiment] 

Next, a description is given of a third 
embodiment of the present invention. FIG. 13 is a 
block diagram showing a configuration of the image 
processing apparatus according to the third 
embodiment of the present invention. Referring to 
FIG. 13, image data is input from the image data 
input part 10, subjected to color correction in a 
color correction unit 200, and output from the image 
data output part 30. 
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The color correction unit 200 includes a 
unit controller 201, an area extraction part 203, a 
characteristic value calculation part 205, a target 
value calculation part 207, a storage part 209, and 
an image data correction part 211. 

The unit controller 201 controls the entire 
color correction unit 200. The area extraction part 
203 extracts a specific color area to be subjected to 
color correction and its background area included in 
the image, data input from the image input part 10, 
which is hereinafter also referred to as the original 
image data, referring to the range of a memory color 
stored in the storage part 209. The characteristic 
value calculation part 205 calculates the 
characteristic values of the extracted specific color 
area and background area. The target value 
calculation part 207 calculates the target value of 
the specific color area in accordance with the 
characteristic values obtained from the 
characteristic value calculation part 205 based on 
the characteristic values and the range of the memory 
color obtained from the storage part 209. The image 
data correction part 211 performs color correction on 
the specific color area extracted in the area 
extraction part 203 based on the calculated target 
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value . 

The image data input part 10 and the image 
data output part 30 have the same functions as in the 
first and second embodiment, and a description 
5 thereof is omitted. 

The storage part 209 stores the chromaticity 
information of memory colors and the color 
information of a preferable color (preferable color 
information) for each memory color. The chromaticity 

10 information is stored as extraction conditions for 

extracting a specified memory color from the original 
image data. The memory colors are colors memorized 
by a man, such as the colors of skin, sky, ocean, 
river, grass and trees, and snow. In the below- 

15 described case, the range of chromaticity coordinates 
for extracting a memory color and the range of 
chromaticity coordinates of a preferable memory color 
are preset as the extraction conditions for 
extracting the memory color. Alternatively, it is 

20 also possible to preset only specific ranges of 
chromaticity coordinates . 

Further, chromaticity is not limited to the 
below-described chromaticity according to the YCC 
color space. Xy chromaticity, and chromaticity using 

25 an RGB ratio are also employable. 
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The specific color subjected to correction 
may be specified by a user as described in the first 
embodiment. In the following description, the 
specific color is the memory color of a human skin 
5 color. However, the present invention is also 
applicable in the case of other memory colors. 

Area extraction part 203 : 

First, the R, G, and B data (R, B, G) of 
each pixel input from the image data input part 10 is 
10 converted into YCC data ( YY , C lf C 2 ) by: 
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where an through 333 are, for instance, constants as 
15 follows : 

0.5866 0.1145 " 
-0.3312 0.5000 
_ 0.4183 -0.0816 

Further, chroma (C) and hue angle (H) are 
20 calculated from the obtained Cj and C2 by: 
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c = Vc, 2 +c 2 2 

- (17) 

// = tan" 1 (C 1 /C 2 ) 

5 Next, a description is given, with reference 

to FIG. 14, of a method of extracting the specific 
color area (skin-colored area) existing in the image 
data from the obtained chromaticity information. FIG. 
14 shows YCC color space coordinates in the Ci-C 2 

10 plane. The distance and the angle from Point O in 
the space represent chroma (C) and hue angle (H) , 
respectively. The data of each pixel of the original 
image data is converted from chromaticity coordinate 
values of the RGB color space into chromaticity 

15 coordinate values of the YCC color space by Equation 
(16) . 

Next, the converted image data is compared 
with the range of the skin color (specific color) 
prestored in the storage part 209. The range of the 

20 skin-colored area may be specified by a user. That 
is, as shown in FIG. 15A, the center C±j of the image 
data is specified, and the quantities r± and rj along 
the vertical and horizontal axes, respectively, of an 
ellipsoid centered on Point C±j are specified. As a 

25 result, a substantial range of the skin-colored area 



can be determined. 

The background area is defined as an area 
formed around the skin-colored area in an ellipsoid 
obtained by increasing the range (dimensions) of the 
skin-colored area by several times of the length of 
the short-axis. 

The range of the skin-colored area and the 
background area obtained by the above-described 
processing are shown as an S area and an R area, 
respectively, in FIG. 15B. 

Characteristic value calculation part 205 : 

The characteristic value calculation part 
205 calculates the characteristic values of the skin- 
colored area and the background area extracted from 
the original image data. The characteristic values 
are calculated by the following two methods, for 
instance. 

(a) A frequency distribution is calculated 
with respect to the mapped points of each of the YY, 
Ci , and C2 planes into which each pixel point 
belonging to the skin-colored area of the original 
image data is mapped by YCC conversion. FIG. 16 is a 
schematic graph of the frequency distribution of the 
luminance YY plane. The horizontal axis represents 
luminance YY , and the vertical axis represents the 
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10 



15 



frequency f (%) of luminance. 

Then, a standard deviation (cr. Syy) in the 
luminance of the skin-colored area is calculated from 
a luminance (value) YYj at a luminance point j and the 
average of the luminance values (a luminance average) . 
Likewise, a standard deviation in each of the Ci and 
C 2 planes is calculated. The standard deviations are 
calculated by: 



where YY , C, , and C 2 are averages in the YY, Ci, and 
C2 planes, respectively, and n is a series of 
frequency distribution in each plane, and is a 
dynamic range that varies dynamically. 

Likewise, standard deviations are calculated 
for the background area by: 
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where 2^, C, , and C 2 are averages in the YY , Ci , and 
C2 planes, respectively, and n is a series of 
frequency distribution in each plane, and is a 
dynamic range that varies dynamically. 

The calculated standard deviations <J. Syy, a 
Scl, <J. Sc2 , (J. Ryy, cr m Rcl , and cr. Rc2 are 
determined as the characteristic values of the 
original image data. 

According to this embodiment, the standard 
deviations of the original image data are employed as 
its characteristic values. However, a variety of 
statistics such as an average, a median, and a mode, 
are also employable. 

(b) A frequency distribution is calculated 
with respect to the mapped points of each of the YY , 
C, and H planes into which each pixel point belonging 
to the skin-colored area of the original image data 
is mapped after being subjected to YCC conversion and 
further converted into chroma (C) and hue (H) by 
Equation (17) . FIG. 17 is a schematic graph of the 
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frequency distribution of the chroma (C) plane. The 
horizontal axis represents chroma (C) , and the 
vertical axis represents the frequency f (%) of 
chroma (C) . 

Then, a standard deviation (p. Sc) in the 
chroma of the skin-colored area is calculated from a 
chroma (value) Cj at a chroma point j and the average 
of the chroma values (a chroma average) . Likewise, a 
standard deviation in each of the hue H and luminance 
YY planes is calculated. The standard deviations are 
calculated by: 



lY" XYY-YY.f / 

\Y" (c-c,) 2 / 

\Y" (H-Hj) 2 / 

where C , H , and YY are averages in the C, H, and YY 
planes, respectively, and n is a series of frequency- 
distribution in each plane, and is a dynamic range 
that varies dynamically. 

Likewise, standard deviations are calculated 
for the background area by: 
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where C , H, and IT are averages in the C, H, and YY 
planes, respectively, and n is a series of frequency 
distribution in each plane, and is a dynamic range 
that varies dynamically. 

10 The calculated standard deviations £>. Syy , p. 

Sc, p. Sh, p. Ryy, p. Rc, and p. R h are determined as 
the characteristic values of the original image data. 

According to this embodiment, the standard 
deviations of the original image data are employed as 

15 its characteristic values. However, a variety of 

statistics such as an average, a median, and a mode, 
are also employable. 

Target value calculation part 207 : 
The target value calculation part 207 

20 calculates the chromaticity information of a target 
value indicating a color to which the color of the 
skin-colored area of the original image data is 
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corrected . 

First, the representative color of each area 
is obtained. Any point in the area is employable as 
the representative color of the area. According to 
5 this embodiment, principal component analysis is 
performed on each area, and the center of the 
obtained principal component axis is determined as 
the representative color of the area. 

Each pixel point of the skin-colored area 

10 extracted from the original image data by the area 
extraction part 203 is subjected to YCC conversion. 
Principal component analysis is performed on the 
skin-colored area after the YCC conversion. The 
center of the obtained principal component axis 

15 (Point S in FIG. 14) is determined as the 

representative color of the skin-colored area. 

Likewise, each pixel point of the background 
area extracted from the original image data by the 
area extraction part 203 is subjected to YCC 

20 conversion. Principal component analysis is 

performed on the background area after the YCC 
conversion. The center of the obtained principal 
component axis (Point R in FIG. 14) is determined as 
the representative color of the background area. 

25 Further, the chromaticity point (Point P in 
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FIG. 14) of a preferable color for the skin color is 
read from the storage part 209. 

Next, the chromaticity point (Point Q) of 
the target value is calculated from Points R and P 
and the characteristic values of the skin-colored 
area and the background area obtained in the 
characteristic value calculation part 103. 

For instance, the distance \ PR\ between 
Points P and R is calculated by: 

\P&\ = y/(C ]p - C lR ) 2 + (C 2P - C 2R ) 2 -(22) 

where Ci P and C^p are the coordinate values of Ci and 
C 2 , respectively, at Point P, and Cm and C 2 r are the 
coordinate values of Ci and C 2 , respectively, at Point 
R . 

Next, Point Q (distant from Point P by Th) 
on the segment PR connecting Points P and R is 
calculated as a target value for correcting the 
preferable skin color in accordance with the color of 
the background area. 

Next, a description is given of a method of 
calculating a correction for each of the above- 
described methods (a) and (b) of calculating the 
characteristic values the skin-colored area and the 
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background area according to the characteristic value 
calculation part 205. 

A method of calculating a correction (Th) 
based on the characteristic values according to the 
above-described method (a) : 

The relationship between corrections and the 
characteristic values of the skin-colored area and 
the background area is prestored in an LUT . A 
correction Th corresponding to the characteristic 
values of the skin-colored area (cr. Syy , cr. Scl , and 
a. Sc2) and the characteristic values of the 
background area (cr. Ryy , cr. Rcl , and cr. Rc2) 
calculated by the method (1) is calculated, referring 
to the LUT. 

For instance, FIG. 18 shows an LUT showing 
the relationship between characteristic value (the 
standard deviation of luminance YY) and correction in 
a general background area. When a characteristic 
value <7. Ryy is given, the corresponding correction 
Th is calculated. 

That is, when the luminance of the 
background area has a broad distribution as in the 
case of, for instance, a landscape image where a 
large quantity of luminance information is employed, 
the calculated correction Th is low. On the other 



hand, in the case of, for instance, a portrait of a 
single color background, the calculated correction Th 
is high. 

That is because when the information of the 
background area is small in quantity, a correction is 
increased, and when the background area has a large 
quantity of information, a correction is reduced 
since emphasis is not put on the reproducibility of 
the skin color. 

According to this embodiment, the luminance 
distribution of the background area is employed. 
However, this is only an example, and the luminance, 
Ci, and C2 components of the background area, and the 
luminance, Ci , and C 2 components of the skin-colored 
area is employable individually or in an appropriate 
combination . 

A method of calculating a correction (Th) 
based on the characteristic values according to the 
above-described method (b) : 

The relationship between corrections and the 
characteristic values of the skin-colored area and 
the background area is prestored in an LUT . A 
correction Th corresponding to the characteristic 
values of the skin-colored area (p. Syy , p. Sc, and p. 
Sh) and the characteristic values of the background 



area (p. Ryy , P- Rc , and p. Rh) calculated by the 
method (1) is calculated, referring to the LUT . 

For instance, FIG. 19 shows an LUT showing 
the relationship between characteristic value (the 
standard deviation of chroma C) and correction in a 
general skin-colored area. When a characteristic 
value p. Sc is given, the corresponding correction Th 
is calculated. 

That is, when the percentage of the skin- 
colored area in the original image data is high, the 
calculated correction Th is high. In the opposite 
case, the calculated correction Th is low. 

This is because when the skin-colored area 
is wide, emphasis is put on the reproducibility of 
the skin color, and when the skin-colored area is 
small, the influence of the reproducibility of the 
skin color over image quality is limited. 

According to this embodiment, the chroma 
distribution of the skin-colored area is employed. 
However, this is only an example, and the luminance, 
chroma, and hue components of the skin-colored area 
and the luminance, chroma, and hue components of the 
background area is employable individually or in an 
appropriate combination . 

The method of calculating the correction Th 
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is not limited to a method that calculates the 
correction Th directly by previously calculating the 
correction of a preferable color for a characteristic 
value and storing the relationship in a memory such 
as an LUT . The correction Th may be calculated from 
a function expressing the relationship between the 
correction of a preferable color and a characteristic 
value. This makes it possible to realize a 
configuration that performs a variety of processing. 
Image data correction part 211 : 
The image data correction part 211 performs 
color correction by shifting the representative color 
(Point S) of the skin-colored area (S range) to the 
target value (Point Q) and shifting the other 
chromaticity points included in the skin-colored area 
in the same manner in the YCC color space (FIG. 14) . 

Referring to FIG. 14, with respect to each 
chromaticity point S± in the skin-colored area (S 
range), a chromaticity point S±' to which the 
chromaticity point S± is shifted by a distance |S£?| in 
the direction from Point S to Point Q is calculated. 
Then, the chromaticity points S± and S±' are inversely 
converted from YCC color space values into RGB color 
space values by Eguation (16) . The pixel data of the 
original . image data which pixel data has the same 
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value as the RGB value into which the chromat icity 
point Si is inversely converted is replaced by the RGB 
value into which the chromaticity point S±' is 
inversely converted. This processing is performed on 
5 every chromaticity point S± included in the skin- 
colored area (S range) , thereby correcting the skin 
color of the original image data. 

In the above-described case, the pixels in 
the skin-colored area are simply shifted uniformly by 

10 the distance | SQ\ . Alternatively, the pixels may be 
moved based on a predetermined function that 
decreases a correction for a pixel in the specific 
color area as the position of the pixel comes closer 
to the background area. 

15 According to the above-described 

configuration, a specific color (for instance, a 
human skin color) may be corrected in accordance with 
its peripheral area so that a preferable color is 
always obtained. 

20 

[ Fourth Embodiment] 

A description is given of a fourth 
embodiment of the present invention. 

In the fourth embodiment, the color 
25 correction unit 200 of the third embodiment obtains a 
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preferable color by reading the attribute information 
of an image. 

FIG. 20 is a block diagram showing a 
configuration of the image processing apparatus 
according to the fourth embodiment of the present 
invention. Referring to FIG. 20, image data is input 
from the image data input part 10, subjected to color 
correction in the color correction unit 200 , and 
output from the image data output part 30. 

The color correction unit 200 includes the 
unit controller 201 , the area extraction part 203, 
the characteristic value calculation part 205, the 
attribute information obtaining part 112, the target 
value calculation part 207, the storage part 209, and 
the image data correction part 211. 

The unit controller 201 controls the entire 
color correction unit 200. The area extraction part 
203 extracts a specific color area to be subjected to 
color correction and its background area included in 
the image data input from the image input part 10, 
which is hereinafter also referred to as the original 
image data, referring to the range of a memory color 
stored in the storage part 209. The characteristic 
value calculation part 205 calculates the 
characteristic values of the extracted specific color 
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area and background area. The attribute information 
obtaining part 112 obtains the attribute information 
of the original image data. The target value 
calculation part 207 calculates the target value of 
the specific color area in accordance with the 
characteristic values obtained from the 
characteristic value calculation part 205 based on 
the characteristic values, the range of the memory 
color obtained from the storage part 209, and the 
attribute information of the original image data 
obtained from the attribute information obtaining 
part 112. The image data correction part 211 
performs color correction on the specific color area 
extracted in the area extraction part 203 based on 
the calculated target value. 

In FIG. 20, the elements having the same 
functions as in the third embodiment (FIG. 13) are 
referred to by the same numerals, and with respect to 
those elements, only a difference from the third 
embodiment is described. 

Target value calculation part 207 : 
A description is given, with reference to 
FIG. 21, of a method of calculating a correction (Th) 
FIG. 21 is a. correction factor table showing a 
correction factor for the correction (Th) of the 



skin-colored area with respect to a given scene 
capture type value. Image data correction is 
performed using a value obtained by multiplying the 
correction Th obtained by the target value 
calculation part 207 of the third embodiment by the 
correction factor . 

That is, when a portrait is captured (this 
case corresponds to Type 2 of FIG. 21) , the 
correction factor is calculated to be 0.5. That is, 
in this case, the correction factor is calculated to 
be a value higher than that for STANDARD 
(corresponding to Type 0 of FIG. 21). This is 
because when a portrait is captured, a correction is 
increased since emphasis is put on the 
reproducibility of a skin-colored area in the image 
data . 

A method of calculating the correction 
factor is not limited to the above-described method 
that refers to the correction factor table. The 
correction factor may also be suitably calculated by 
a prepared function based on the image attribute 
information. Further, not only a luminance component 
but also a chroma or hue component is employable as 
the correction Th . 

Next, a description is given, with reference 



to FIG. 22, of a procedure for correcting the 
original image data in the image processing apparatus 
according to the third and fourth embodiments . 

First, in step S20 of FIG. 22, image data is 
input from outside via a network or from an apparatus 
such as a hardware disk drive, a flexible disk drive, 
an optical recording medium reader, a scanner, or a 
digital camera. When the input image data is an 
analog signal, the image data is converted into a 
digital signal. Thereafter, the image data is 
subjected to predetermined digital processing, and is 
output as digital signals of R (red) , G (green) , and 
B (blue) . 

Next, in step S21, the data of each pixel of 
the original image data is converted from 
chromaticity coordinate values of the RGB color space 
into chromaticity coordinate values of the YCC color 
space by Equation (16) . The converted image data is 
compared with the range of the skin color (specific 
color) prestored in the storage part 209. Thereby, 
the specific color area (skin-colored area) existing 
in the original image data is extracted. An area 
formed around the skin-colored area in an ellipsoid 
obtained by increasing the range (dimensions) of the 
skin-colored area by several times of the length of 



the short-axis is defined as the background area of 
the specific color area. 

Next, in step S22, the characteristic values 
of the extracted skin-colored area and background 
area are calculated. The characteristic values may 
be calculated by one of the following two methods: 

(a) A frequency distribution is calculated 
with respect to the mapped points of each of the YY, 
Ci , and C 2 planes into which each pixel point 
belonging to the skin-colored area of the original 
image data is mapped by YCC conversion. The standard 
deviations in the planes of the skin-colored area are 
calculated by Equation (18) to be employed as the 
characteristic values of the skin-colored area. 
Likewise, the standard deviations in the planes of 
the background area are calculated by Equation (19) 
to be employed as the characteristic values of the 
background area. 

(b) A frequency distribution is calculated 
with respect to the mapped points of each of the YY, 
C, and H planes into which each pixel point belonging 
to the skin-colored area of the original image data 
is mapped after being subjected to YCC conversion and 
further converted into chroma (C) and hue (H) by 
Equation (17). The standard deviations in the planes 
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of the skin-colored area are calculated by Equation 
(20) to be employed as the characteristic values of 
the skin-colored area. Likewise, the standard 
deviations in the planes of the background area are 
5 calculated by Equation (21) to be employed as the 
characteristic values of the background area. 

Next, in step S23, the chromaticity 
information of a target value indicating a color to 
which the color of the skin-colored area of the 

10 original image data is corrected is calculated. 

First, principal component analysis is 
performed on each of the skin-colored area and the 
background area. The centers of the obtained 
principal component axes of the skin-colored area and 

15 the background area are obtained as their respective 
representative colors (Points S and R in FIG. 14). 

Next, the chromaticity point (Point P in FIG. 
14) of a preferable color for the specific color 
(skin color) is read out from the storage part 209. 

20 The chromaticity point (Point Q, distant from Point P 
by Th) of the target value is calculated from Points 
£ and P and the characteristic values of the original 
image data . 

If the image attribute information added to 
25 the original image data has been obtained at this 



point, a correction factor a according to the image 
attribute information is obtained from the correction 
factor table. The correction Th is multiplied by the 
correction factor a to be determined as the final 
correction . 

Then, in step S24, color correction is 
performed by shifting the representative color (Point 
S) of the skin-colored area (S area) to the target 
value (Point Q) and shifting the other chromaticity 
points included in the skin-colored area in the same 
manner in the YCC color space. 

With respect to each chromaticity point Si 
in the skin-colored area (S range) , a chromaticity 
point S±' to which the chromaticity point S± is 
shifted by a distance \ SQ\ in the direction from 
Point S to Point Q is calculated. Then, the 
chromaticity points S± and Si' are inversely converted 
from YCC color space values into RGB color space 
values by Equation (16) . The pixel data of the 
original image data which pixel data has the same 
value as the RGB value into which the chromaticity 
point Si is inversely converted is replaced by the RGB 
value into which the chromaticity point Si' is 
inversely converted. This processing is performed on 
every chromaticity point Si included in the skin- 
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colored area(S range), thereby correcting the skin 
color of the original image data. 

In step S25, the corrected image data 
obtained by performing color correction on the 
5 original image data is output to a printer, a multi- 
function apparatus, a facsimile machine, or a storage 
device such as a hard disk unit or an optical 
recording medium recorder. Alternatively, the 
corrected image data may be output to outside via a 
10 network. 



[Other embodiments] 

Any of the following configurations is 
employable in the present invention for performing 
15 desired color correction on image data. 

(a) A computer is provided between an image 
input device and an image output device. The 
computer performs color correction on read image data, 
and then, the corrected image data is output. 
20 (b) A color correction unit performing 

desired color correction is incorporated into a 
digital camera. An image is displayed on a print or 
a liquid crystal panel using the converted image data, 
(c) A color correction unit performing 
25 desired color correction is incorporated into a 
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printer. Image data input directly is subjected to 
the desired color correction, and is printed out. 

Further, the functions of the apparatus of 
each of the above-described embodiments are 
5 previously written to a recording medium such as a 
ROM or a CD-ROM as a program. The recording medium 
is loaded into an image processing apparatus so that 
the program is executed by a micro processor. 
Thereby, the objects of the present invention are 
10 achieved. 

In this case, the program read out from the 
recording medium realizes the above-described 
embodiments. Accordingly, the program and the 
recording medium on which the program is recorded are 

15 also included in the present invention. 

The recording medium may be a semiconductor 
medium such as a ROM or a nonvolatile memory card, an 
optical medium such as a DVD, a MO disk, an MD , or a 
CD-R, or a magnetic medium such as a magnetic tape or 

20 a flexible disk. 

Alternatively, the program may be stored in 
the storage device of a server computer and supplied 
directly therefrom via a communication network such 
as the Internet. In this case, the storage device of 

25 the server computer is also included in the recording 
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medium of the present invention. 

Further, the functions of the above- 
described embodiments are realized by not only 
executing the loaded program, but also performing 
5 processing in cooperation with an operating system or 
another application program based on the instructions 
of the program. 

Further, in the case of providing use by an 
ASP (application service provider) that stores the 

10 above-described program in the storage device of a 
server computer, such as a magnetic disk, executes 
the program upon receiving an execution instruction 
from a user computer connected via a communication 
line such as the Internet, and returns the execution 

15 results to the user computer, the storage device of 

the server computer and the program are also included 
in the present invention. 

By thus distributing the functions according 
to the above-described embodiments in the form of a 

20 program, costs, portability, and versatility can be 
improved. 

The present invention is not limited to the 
specifically disclosed embodiments, and variations 
and modifications may be made without departing from 
25 the scope of the present invention. 
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The present application is based on Japanese 
priority applications No. 2002-348384, filed on 
November 29, 2002, and No. 2003-376443, filed on 
November 6, 2003, the entire contents of which are 
5 hereby incorporated by reference. 



